What is the Teacher’s Role in Robot Programming by Demonstration? Toward Benchmarks for Improved Learning

Robot programming by demonstration (RPD) covers methods by which a robot learns new skills through human guidance. We present an interactive, multimodal RPD framework using active teaching methods that places the human teacher in the robot’s learning loop. Two experiments are presented in which observational learning is first used to demonstrate a manipulation skill to a HOAP-3 humanoid robot by using motion sensors attached to the teacher’s body. Then, putting the robot through the motion, the teacher incrementally refines the robot’s skill by moving its arms manually, providing the appropriate scaffolds to reproduce the action. An incremental teaching scenario is proposed based on insights from various fields addressing developmental, psychological, and social issues related to teaching mechanisms in humans. Based on this analysis, different benchmarks are suggested to evaluate the setup further. In a robot programming by demonstration (RPD) framework, a robot learns new skills through the help of a human instructor (Billard & Siegwart, 2004). Traditionally, RPD tends to consider the human user as an expert model who performs a task while the robot observes passively the demonstration (Ikeuchi & Suchiro, 1992; Kuniyoshi, Inaba, & Inoue, 1994). However, in humans, teaching is a social and bidirectional process in which teacher and learner are both active. Instead of considering the teacher solely as a model of successful expert behavior, recent work has referred to the teacher-learner couple as a We gratefully acknowledge Chrystopher L. Nehaniv, Aris Alissandrakis, Joe Saunders, Nuno Otero and Kerstin Dautenhahn for the useful exchanges of thoughts concerning social cues and feedback as well as the user experience and evaluation issues tackled by the Cogniron project. We would also like to acknowledge the four anonymous reviewers for their very useful comments on an earlier version of the manuscript. The work described in this paper was supported in part by the Secretariat d’Etat a l’Education et la Recherche Suisse (SER), under Contract FP6-002020, Integrated Project Cogniron of the European Commission Division FP6-IST Future and Emerging Technologies, and by the Swiss National Science Foundation, through grant 620-066127 of the SNF Professorships program. WHAT IS THE TEACHER’S ROLE IN ROBOT PROGRAMMING BY DEMONSTRATION? 2 Figure 1. Different modalities are used to convey the demonstrations and scaffolds required by the robot to learn a skill. The user first demonstrates the whole movement using motion sensors (left) and then helps the robot refine its skill by kinesthetic teaching (right), that is, by embodying the robot and putting it through the motion. team that engages in joint problem solving (Breazeal et al., 2004) and uses active teaching methods to put the human teacher “in the loop” of the robot’s learning (Dautenhahn, 1998). In previous work, we developed a probabilistic framework for extracting the relevant components of a task by observing multiple demonstrations of it (Calinon, Guenter, & Billard, 2007). The system is based on a probabilistic encoding of several demonstrations provided to the robot to generalize the learned skills to different contexts using a Gaussian mixture model (GMM). From these experiments, we noticed the importance of adding social components to the teaching paradigm, not only in the user interface but also in the teaching process (Calinon & Billard, 2007). In contrast to the traditional RPD approach, we thus adopted the perspective that the transfer of skills can draw advantages of several psychological factors that the user might encounter during teaching. Thus, the teacher is no longer considered solely a model of expert behavior but becomes an active participant in the learning process (Yoshikawa, Shinozawa, Ishiguro, Hagita, & Miyamoto, 2006). First, we suggest using different modalities to produce the demonstrations (Figure 1). We then suggest following an incremental learning approach to allow the teacher to gradually see the results of the demonstrations.